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(U)  Detection  of  Different  Target  Types  In  Realistic  Terrain,  by  G. 
E.  Corrick,  Hughes  Aircraft  Company  China  Lake,  Calif.,  Naval  Weapons 
Center,  March  1979.  36  pp.  (NWC  TP  6107,  publication 

-UNCLASSIFIED.) 

(UV'The  effects  of  targel  and  terrain  characteristics  on  visual 

air-to-ground  target  acquisition  were  studied.  Five  target  types^a  portable 
bridge;  a portable  bridge  with  adjacent  anti-aircraft  artillery  (AAA);  a 
surface-to-air  missile  (SAM)  site;  a petruleun),  oil,  and  lubricants  (POL) 
supply  dump,'  and  a group  of  three  tanksjrrwere  embedded  Into  oblique 
aerial  photos  of  real  terrain  at  a simulated  slant  range  of  1.6  km. 

Background  scenes  were  selected  to  represent  desert,  desert/mountain,  and 
rural  terrain.  . 

(U)  The  Results  of  a search  experiment  showed  that  target  type  was 

the  most  important  factor  in  determining  acquisition  performance, 

accounting  for  up  to  40%  of  the  experimental  variance.  The  relative 
detectabilities  of  the  targel  were  found  to  group  so  that  the  bridge  alone, 
the  bridge  with  AAA  site,  and  the  SAM  site  were  significantly  easier  to 
detect  than  the  three  tank  group  or  POL  site.  1 -beSfc  ^fsults  are  discussed 
in  terms  of  the  detectability  of  a target  as  related  to  the  constraints  on 
its  possible  scene  location. 

(V) '  Background  was  shown  to  be  a significant  effect,  but  accounted 
for  about  one-fourth  as  much  of  the  variance  as  did  target  type.  These 
results  are  compared  to  a previous  similar  study  using  multiple 
configurations  of  a single  target  type  where  background  characteristics 
were  ^hown  to  account  for  mote  variance.  The  implications  of  these 
results  for  target  acquisition  modeling  are  discussed. 
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INTRODUCTION 


This  interim  report  describes  an  experiment  conducted  to  provide 
data  to  augment  a target  acquisition  model  which  is  part  of  a larger 
algorithm  designed  to  compute  the  probability  of  an  aircrew's  success- 
ful delivery  of  air-to-ground  weapons.  1 The  present  experiment 
focuses  on  the  way  in  which  target  and  terrain  types  influence  visual 
target  acquisition  performance  and  it  is  an  extension  of  a previous 
study  demonstrating  the  importance ^of  both  target  number  and  its  inter- 
action with  terrain  characteristics."  Specifically,  the  Olzak  experi- 
ment dealt  with  detection  of  a single  target  type,  a tank,  presented  in 
different  numbers  and  configurations.  The  present  experiment,  on 
the  other  hand,  is  concerned  with  detection  of  wholly  different  target 
types,  viz.,  an  SA-6  surface-to-air-missile  (SAMI  site;  a petroleum, 
oil,  and  lubricants  (POL)  supply  dump;  a group  of  three  tanks;  a por- 
table bridge;  and  a bridge  with  surrounding  anti-aircraft  artillery 
(AAA).  Target  types  such  as  these  obviously  differ  both  in  the  number 
of  elements  comprising  a target  and  in  configuration.  The  purpose  of 
this  experiment  is  to  gather  data  on  the  relative  detectabilities  of  these 
different  target  types  in  addition  to  investigating  how  the  terrain  type 
in  which  the  target  is  located  influences  the  relative  detectability  of 
these  targets.  These  data,  combined  with  those  of  the  previous  report 
by  Olzak  will  bo  used  in  the  formulation  of  a predictive  model  of  target 
acquisition  performance  based  on  fuller  appreciation  of  the  interactive 
effects  of  target  and  terrain  types.  This  modeling  effort  constitutes 
Phase  II  of  the  present  contract  and  will  be  presented  In  a subsequent 
report. 


TARGET  ACQUISITION  MODELING 

The  development  of  good,  predictive  mathematical  models  of  the 
target  acquisition  process  has  obvious  practical  utility  for  military 


Naval  Weapons  Center.  Launch  Opportunity  for  Air-to-Ground 
Visually  Guided  Weapons^  'R.  A.  'Erickson  and  t.  T7  Burge. 

China  Lake,  Calif.  , NWC,  January  1978.  (NWC  TP  6005,  publica- 
tion UNCLASSIFIED.  ) 

) 

"Naval  Weapons  Center.  Detection  of  Multiple -Vehicle  Targets  in 
Realistic  Terrain.  L.  A.  Olzak,  Display  Systems  Department, 
Hughes  Aire  raft  Company.  China  Lake,  Calif.,  NWC,  July  1978. 
(NWC  TP  6061,  publication  UNCLASSIFIED.  ) 
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analysts  and  planners  and  as  such  has  been  actively  pursued  tor  more 
than  10  years.  The  resulting  models  are  characterized  by  a number 
of  different  approaches  and  initial  assumptions  and  vary  widely  in 
complexity.  A detailed  review  of  the  models  is  not  required  here, 
having  been  excellently  covered  elsewhe re.  3 , 4,  5 ^ a relevant  point 
that  emerges,  however,  is  that  both  target  type  and  terrain  character- 
istics have  been  relatively  ignored  in  target  acquisition  modeling.  If 
included  in  a model,  target  effects  are  usually  specified  only  by 
overall  target  dimensions  and  for  the  number  of  elements  in  the  tvrget. 
Target  type,  per  se  is  usually  not  treated  and  hence  a whole  range  of 
local  area  context  effects  are  disregarded.  Terrain  classification, 
when  explicitly  dealt  with  in  models,  Is  typically  treated  by  a global, 
unidimensional  metric  such  as  scene  complexity,  "busyness,  " or 
heterogeneity.  Such  measures,  which  assess  scene  characteristics 
in  isolation  from  information  about  target  type,  are  not  good  target 
acquisition  performance  predictors.  A good  example  of  this  situation 
is  given  hy  the  study  of  Zaitzeff”,  which  used  photos  of  real  targets 
in  real  scenes.  In  this  experiment  a global  subjectively  measured 
heterogeneity  measure  was  shown  to  be  a reliable  measure  of  scene 
complexity:  that  is,  it  was  able  to  consistently  discriminate  scenes. 
However,  this  measure  was  shown  to  account  for  a small  portion  of 
the  variance  in  the  flrv»l  regression  equation  and  hence  was  not  con- 
sidered a good  predictor  of  target  acquisition  performance. 


C.  P.  Greening.  "Mathematical  modeling  of  air-to-ground  target 
acquisition,'  Human  Factors,  Vol.  Id  No.  2 (April  1876), 
pp . 111-147. 
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Naval  Weapons  Center.  Target  Acquisition  Model  Evaluation: 

Final  Summary  Report,  by  C.  P.  Greening,  Autoneties  Division, 
Rockwell  international.  China  Lake,  Calif.,  NWC,  June  1973. 

( N W C TP  553b,  publication  UNCLASSIFIED.  1 
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Naval  Weapons  Center.  Target  Acquisition  Model  Evaluation:  Part  2. 
A Review  of  British  Target  Acquisition  Models,  by  C.  P.  Greening, 
Autoneties  Division,  Rockwell  International.  China  Lake,  Calif.  , 
NWC,  August  1974.  (NWC  TP  5536,  Part  2,  publication 
UNCLASSIFIED.  ) 

The  Boeing  Company.  Target  Background  Scaling  and  its  Impact  on 
the  Prediction  of  Aircrew  Target  Acquisition  Performance,  by  L~  P. 
Zaitsoff,  Aerospace  Group,  Boeing  Company,  Seattle,  Wash.  , 
December  1971.  (AD737693  D1  80- 1 4 1 56-1 , publication 
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Olzak  explicitly  tested  the  adequacy  of  the  heterogeneity  measure 
by  testing  detection  performance  by  embedding  targets  in  real  scene 
photographs.  In  this  study  the  targets  consisted  of  a single  tank,  and 
groups  and  convoys  of  three  or  nine  tanks.  The  scene  heterogeneity 
measure  was  found  to  have  high  internal  validity  — scenes  rated 
equivalent  in  heterogeneity  produced  equivalent  performance.  How- 
ever, heterogeneity  did  not  predict  performance  well:  a correlation 
of  only  0.  18  was  obtained  between  heterogeneity  measures  and  detec- 
tion performance. 

Olzak**  did  find,  however,  that  target  characteristics  provided  good 
performance  predictors.  Performance  varied  directly  with  target 
number,  i.  e.  , one,  three,  or  nine  tanks,  while  target  configuration 
was  important  only  for  the  nine  tank  condition.  The  location  within  a 
scene  of  a target  was  also  important,  but  only  for  the  single  tank  con- 
dition where  performance  was  better  if  the  tank  was  located  on  or 
near  a road.  It  was  suggested  that  for  increasing  target  numbers 
roads  become  less  important  cues  to  target  location.  These  results 
highlight  the  importance  of  scene  and  target  characteristics  in  predict- 
ing target  acquisition  performance.  The  purpose  of  the  present 
experiment  is  to  investigate  such  effects  with  different  target  types. 
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METHOD 


Five  target  types  were  investigated  in  this,  study:  l)  a group  of 
three  tanks:  2)  an  SA-6  SAM  site:  3)  a POL  supply  dump;  4)  a bridge; 
and  5)  a bridge  with  mobile  AAA  units  adjacent.  Each  of  these  target 
types  was  embedded  in  six  aerial  oblique  terrain  background  photo- 
graphs, representing  two  each  of  three  generic  terrain  types  — rural, 
desert,  and  desert /mountain  terrain.  Subjects  were  assigned  to  one 
of  five  groups  defined  by  target  type:  each  subject  thus  saw  all  back- 
grounds but  only  one  target  type.  The  subjects  were  allowed  30  sec- 
onds to  search  the  scenes  for  the  pre-briefed  target  type,  and  dependent 
measures  of  time  to  detection  and  probability  of  correct  detection  were 
recorded. 


EXPERIMENTAL  DESIGN 

This  study  manipulated  two  variables,  target  type  and  background, 
with  repeated  measures  on  the  background  factor,  and  independent 
measures  on  the  target  factor.  The  design  is  variously  referred  to 
as  a mixed  repeated-measure  or  split-plot  design.  Because  the  five 
groups  of  subjects  each  viewed  a single  target  type,  the  subjects  factor 
is  nested  within  the  targets  factor.  Thus,  the  between  groups  factor 
was  target  type,  sampled  at  five  levels  and  the  within  groups  factor 
was  background,  sampled  at  six  levels. 

This  design  is  somewhat  different  from  the  one  employed  by  Olzak. 
In  that  study  the  within  groups  factor  was  a composite  of  three 
variables  — scene  complexity,  complexity  replications,  and  local 
context.  The  scene  complexity  measure  was  an  adaptation  of  the 
heterogeneity  measure  developed  by  Ciavarelli,  Wachter,  and  Lee7 
and  consisted  of  paired-comparison  ratings  of  scene  complexity  or 
busyness.  Because  this  measure  was  found  to  correlate  poorly  with 
performance  in  the  Olzak  study,  it  was  not  included  in  the  present 
experiment.  This,  of  course,  also  obviated  the  need  for  a complexity 
replications  factor.  The  final  within  groups  factor  from  the  Olzak 
study,  local  context,  was  a measure  of  whether  a target  appeared  on 
or  near  roads  as  opposed  to  off  roads.  This  factor  is  quite  reasonable 


Naval  Weapons  Center.  Terrain  Classification  Study,  by  A.  Ciavarelli, 
L.  Wachtcr,  and  W.  Leel  Research  and  Engineering  Division,  Boeing 
Aerospace  Company.  China  Lake,  Calif.,  NWC,  May  1975.  (NWC 
TP  5 7 (S A , publication  UNCLASSIFIED.) 
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tor  Ibing  the  position  of  tanks  as  in  Olzak's  study,  but  lacks 

meaning  when  dealing  with  SAM  sites  or  bridges  ns  in  this  experiment. 
An  such  local  context  was  deleted  from  this  study  as  an  experimental 

fa  c t v)  r . 


STIMULUS  MATERIALS 

Thirteen  backgrounds  were  selected  from  the  set  supplied  by  the 
Naval  Weapons  Center  and  which  were  originally  generated  and 
described  by  Ciavarelli,  et  al^.  Of  the  12  backgrounds  six  were 
experimental  scenes,  five  were  training  scenes,  and  two  were  used 
a»  targetless  catch  trials.  In  addition,  five  slides  from  a previous 
study,  one  each  of  the  five  target  configurations,  were  used  for 
preliminary  training  and  to  accustom  the  subjects  to  the  operation  of 
the  apparatus. 

Two  major  constrair.ts  were  placed  on  selection  of  the  experimental 
scenes.  First,  two  each  of  rural,  desert,  and  desert /mountain  terrain 
types  were  desired.  Second,  the  backgrounds  were  required  to  have 
reasonable  locations  in  which  to  embed  all  of  the  five  target  types  used 
in  this  study.  The  latter  constraint  meant  that  road3  or  rivers  were 
needed  for  the  bridge  .argets  am.  a reasonably  sized  open  space  for 
the  SAM  site.  These  constraints  were  relaxed  somewhat  for  the  selec- 
tion of  the  images  for  training  purposes,  as  these  images  were  con- 
sidered of  secondary  importance  to  the  experimental  scenes. 

One  copy  of  each,  of  the  six  experimental  backgrounds  was  embedded 
with  one  of  the  five  target  types,  for  a total  of  30  experimental  scenes. 
Two  copies  of  each  background  used  for  training  were  prepared  and 
each  was  embedded  with  a single  target  type,  again  with  the  constraint 
of  a reasonable  target  location.  All  backgrounds  were  embedded  in  a 
similar  manner,  described  in  the  next  section,  and  all  scenes  repre- 
sented a 30°  field -of- view  (FOV)  with  targets  at  a slant  range  of 
approximately  1.  6 kilometers.  The  slant  range  to  the  target  had  to 
vary  slightly  to  provide  reasonable  target  locations  in  each  background. 


TARGET  EMBEDDING 

An  embedding  procedure  somewhat  differer  from  the  photographic 
embedding  of  model  targets  described  by  Olzak  was  employed  in  this 
study.  This  technique  consisted  of  an  artist  drawing  the  desired  tar- 
gets on  clear  overlays  on  enlargements  of  the  background  scenes, 
superimposing  the  overlay  and  background,  and  then  photographically 
reducing  the  composite  to  the  final  proper  size  for  presentation  in 
35-mm  slide  format. 
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Alter  selection  of  the  background  scenes,  the  original  70-mm  black 
and  white  negatives,  supplied  by  the  Naval  Weapons  Center  and  repre- 
senting 60°  fields  of  view  taken  from  a height  of  approximately 
300  meters,  were  processed  into  film  positives  enlarged  four  times 
and  paper  prints  enlarged  eight  times.  Tracing  overlays  were  placed 
on  the  paper  prints,  and  the  various  target  types  were  sketched  into 
the  desired  location  at  the  proper  scale.  Individual  overlays  were 
made  for  each  ta rget -background  combination  to  provide  the  best  tar- 
get fit  into  each  background.  Using  the  rough  sketch  overlays  as  a 
guide,  an  arti3t  made  refined  ink  drawings  on  clear  acetate  overlays 
affixed  to  the  paper  prints  of  the  backgrounds.  The  artist  was  careful 
to  simulate  correct  depression  angle,  target  size,  shadowing,  and 
internal  modulation  of  the  targets  using  other  elements  of  the  scene 
as  guides. 

The  finished  artist's  overlays  were  placed  on  a high  contrast  white 
background  and  photographically  reduced  by  a factor  of  four  and 
processed  as  negative  transparencies.  These  negatives  were  then 
opaqued  in  the  immediate  target  area  and  then  contact  printed  into 
clear  film  positives.  The  result  was  a transparency  containing  only 
the  target  elements.  These  "target"  film  positives  were  then  posi- 
tioned over  the  previously  generated  four-time  enlarged  background 
film  positives  with  the  target  elements  in  their  proper  location.  The 
resulting  composite  was  processed  to  produce  a contact  film  negative. 

The  target-background  negatives  were  marked  to  give  the  desired 
30°  FOV  and  photographed  as  final  35-mm  film  positives.  These  final 
positives  were  mounted  in  metal  and  glass  slide  mounts  for  use  in 
Carousel  slide  projectors. 


APPARATUS 

The  stimuli  were  presented  via  a two-channel  projection  system 
incorporating  two  Kodak  Carousel  35-mm  slide  projectors,  two 
Gerbrands  shutters  (#G1166),  a half-silvered  mirror  combining  glass, 
and  a rear  projection  screen.  Both  projectors  were  focused  within  a 
0.  61  meter  area  of  the  projection  screen. 

The  resolution  of  the  projection  system,  measured  on  the  rear- 
projection  screen  with  a Buckbee -Mear s resolution  chart,  was 
3.  8 optical  lines/mm  which  is  equivalent  to  193  TV  lines/inch.  A 
control  panel  actuated  the  shutters  and  provided  a digital  timer  whose 
onset  was  synchronous  with  the  opening  of  the  stimulus  shutter.  The 
subject  was  provided  with  a response  key  which  stopped  the  timer, 
and  a stylus  to  indicate  the  location  of  detected  targets'.  A schematic 
diagram  of  the  apparatus  is  shown  in  Figure  I. 
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SUBJECT 


FIGURE  1.  Schematic  Diagram  of  Expe  rimental  Apparatus. 


The  screen  was  set  1.  14  meters  from  the  subject's  eye;  at  this 
distance  the  0.  61  meter  display  subtended  30°  FOV.  A constant  view- 
ing distance  was  maintained  by  a chin  and  forehead  rest. 


SUBJECTS 


The  25  subjects  were  Hughes  Aircraft  Company  employees.  All 
were  screened  to  insure  Snellen  acuity  of  at  least  20/20  with  or  without 
correction.  Thirteen  subjects  had  previously  participated  in  other 
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target  acquisition  experiments;  the  other  12  subjects  hod  never  par- 
ticipated in  a human  factors  experiment  of  any  kind.  Subjects  were 
randomly  as  sinned  to  one  of  the  five  target-type  groups. 


PROCEDURE 

Subjects  were  run  individually  in  20-mlnute  sessions  which  began 
with  an  assessment  of  visual  acuity  by  a Snellen  chart  and  followed  by 
an  explanation  oi  the  task  and  procedure.  The  room  lights  were  dimmed 
and  the  subject  shown  a grid  slide  which  adapted  the.pi  to  approximately 
the  luminance  of  the  experimental  slides,  3.  6 cd/m“.  The  grid  slide 
provided  a fixation  point  to  initiate  trials  and  a set  of  coordinates  to 
locate  targets. 

To  familiarize  the  subjects  with  the  task  and  apparatus,  they  were 
then  shown  five  slides  containing  tank  targets  of  various  numbers  and 
configurations  taken  from  the  Olzak  study.  The  subjects  were  informed 
of  the  number  of  tanks  and  configuration  before  each  of  these  slides  and 
were  permitted  as  much  time  as  needed  to  locate  the  targets.  The 
experimenter  pointed  out  the  location  of  missed  targets. 

Following  this  familiarization  period,  the  subject  was  shown  pic- 
tures of  the  five  target  types  to  be  encountered  on  the  next  training 
slides.  Again  the  experimenter  informed  the  subject  of  the  target  type 
to  be  seen  next,  and  feedback  on  the  actual  location  of  missed  targets 
was  provided.  The  last  two  training  slides  in  this  set  contained  the 
target  the  subject  would  search  for  in  the  experimental  slides,  and  the 
subject  was  so  informed. 

After  the  presentation  of  the  training  slides,  the  six  experimental 
and  two  catch  slides  were  presented  but  without  performance  feedback. 
Upon  completion  of  data  collection,  subjects  were  informed,  if  desired, 
of  their  overall  performance. 

The  conduct  of  training  and  experimental  trials  was  similar.  Initially 
the  subject  fixated  a cross  at  the  center  of  the  grid  slide  for  5 seconds. 
The  experimenter  said  "ready"  and  following  a 1 -second  delay  operated 
the  tachistoscope  controls,  simultaneously  replacing  the  grid  slide  with 
a training  or  experimental  slide  and  starting  a timer.  Subjects  could 
search  for  up  to  30  seconds.  Locating  a target,  the  subject  pressed  a 
button  stopping  the  timer  and  pointed  to  the  middle  of  the  target  con- 
figuration with  a pointer.  Simultaneously,  the  experimenter  re-displayed 
the  grid  slide,  and  the  subject  read  off  the  coordinates  of  the  pointer's 
position.  The  experimenter  then  recorded  the  location  and  detection 
time,  advanced  to  the  next  slide,  and  initiated  the  next  trial. 
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RESULTS  AND  DISCUSSION 


The  two  dependent  measures  taken  in  thia  experiment,  detection 
time  and  probability  o£  detection,  were  analyzed  separately  by  analysis 
of  variance  and  various  post  hoc  comparisons  designed  to  elucidate  the 
nature  of  significant  main  effects  and  the  locus  of  interactions.  The 
dependent  variables  were  analyzed  in  terms  of  the  principal  experi- 
mental factors:  target  effects,  background  effects,  and  the  interaction 
of  these  factors. 

In  performing  the  data  analysis  of  the  detection  times,  a problem 
arises  concerning  how  to  treat  incorrect  trials  or  trials  where  no 
detection  occurred.  Such  trials  are  tantamount  to  missing  data  when 
considering  time  to  detection  and  as  Winer®  points  out  there  is  no 
satisfactory  way  to  treat  missing  data  unless  the  nature  of  the  experi- 
mental response  surface  is  adequately  known.  The  solution  employed 
in  this  experiment  was  to  assign  all  incorrect  responses  the  maximum 
allowable  detection  time  of  3(f  seconds,  and  this  was  done  chiefly  to 
maintain  consistency  in  data  analysis  between  this  study  and  the  experi- 
ment of  Olzak.2  This  procedure,  however,  is  not  without  conse- 
quences. Chief  among  these  is  that  the  distribution  of  detection  times 
becomes  skewed  and  the  variance  of  detection  times  is  increased. 

Thus  the  standard  deviation  of  all  detection  times  is  12.2  seconds  as 
opposed  to  a standard  deviation  of  5.7  seconds  for  only  the  detection 
times  on  correct  trials.  This  brief  discussion  serves  only  to  point 
out  the  problem  and  its  consequences;  however,  in  lieu  of  a more 
rational  way  of  treating  these  cases,  the  analysis  will  incorporate  the 
30  second  time  score  for  missing  data. 


ANALYSIS  OF  VARIANCE 

Tables  1 and  2 present  the  results  for  the  overall  analysis  of  vari- 
ance of  the  time  to  detection  and  probability  of  detection  data,  respec- 
tively. The  results  are  unambiguous.  For  both  dependent  variables, 
the  main  effects,  target  type  and  background,  are  highly  significant, 
as  is  the  interaction  between  these  variables.  The  conclusion  based 
on  this  analysis  is  that  target  type,  background,  and  the  interaction 
between  target  and  background  cause  reliable  differences  in  the  time 
required  for  target  detection  and  the  probability  of  detection. 


8,  B.  J.  Winer.  Statistical  Principles  in  Experimental  Design. 
New  York:  McGraw  Hill,  1962. 
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TABLE  1.  Analysis  ot  Variance  Summary:  Time  to  Detection 
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TABLE  2.  Analysis  of  Variance  Summary:  Probability  of  Detection. 
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While  highly  significant  main  effects  and  interactions  imply  a strong 
statistical  relation  between  the  independent  and  dependent  variables, 
significance  tests  alone  say  nothing  about  the  strength  of  the  statistical 
relation.  Simon  ■ following  the  lead  of  Hays^  advocates  the  use  of  a 
measure  showing  the  strength  of  a significant  effect.  The  traditional 
measure  used  by  Hays  is  u»2  which  essentially  is  a correlation  between 
the  variance  due  to  an  experimental  factor  an<^  the  total  experimental 
variance.  Simon, however,  points  out  that  w*"  is  a strength-of-effect 
measure  for  a population;  for  sample  statistics  Simon  advocates  Eta^ 
which  is  the  ratio  between  the  sum- of- squares  for  the  variable  of 
interest  and  the  total  sum- of- squa re s.  Tables  1 and  2 present  both  u>2 
and  Eta~  calculations.  As  can  be  seen,  is  generally  smaller  than 
Eta2.  In  this  analysis  both  background  and  targets  are  treated  as  fixed 
effects,  meaning  they  are  the  particular  levels  of  these  variables  and 
are  not  samples  from  a larger  population.  Hence,  the  Eta2  measure 
is  more  appropriate  to  describe  the  strength  <:[  effect. 

More  important,  however,  than  the  statistical  particulars  of  the 
strength-of-effect  measures  are  the  results  of  the  calculations.  Both 
dependent  variables  present  the  same  picture:  the  target  type  factor 
accounts  tor  about  five  times  the  experimental  variance  as  background 
type.  This  means  that  target  type  was  a much  stronger  effect  than 
background  in  determining  target  acquisition  performance.  Modeling 
efforts  attempting  to  account  for  target  and  background  effects  thus 
take  this  result  into  consideration  and  the  appropriate  way  to  achieve 
this  is  to  give  a target  type  measure  relatively  more  weight  than 
measures  of  background  characteristics.’'' 

? 

The  Eta”  calculations  also  show,  for  both  dependent  variables,  that 
the  interaction  of  target  and  background  accounts  for  an  appreciable 
amount  of  the  total  experimental  variance.  Given  the  predominance  of 
the  target  main  effect  over  the  background  main  effect,  the  strength  of 
the  interaction  is  probably  due  mainly  to  target  effects. 

.A  significant  interaction  is  usually  interpreted  as  meaning  that  the 
effect  on  performance  of  one  variable  depends  on  the  level  of  the  other 
variable.  The  interaction  in  this  equipment  is  taken  to  mean  that  tar- 
get acquisition  performance  is  determined  chiefly  by  the  target  type 


■‘This  conclusion  will  be  tempered  somewhat  when  a reanalysis  of  the 
data  from  Olzak1  s study  is  presented  below. 

9.  Hughes  Aircraft  Co.  Considerations  for  the  Proper  Design  and 
Interpretation  of  Human  Factor  Engineering  Experiments. 

(3.  w.  Simon,  Culver  City,  Calil.  Decemoer  1971.  HAC  (Tech- 
nical Report  P73-325,  publication  Unclassified). 

10.  \V,  L.  Hays.  Statistic s.  New  York:  Holt,  Rinehart,  and 

Winston,  1963. 


14 


Ti 


NWC  TP  61 07 


being  considered,  but  this  effect  is  modulated  somewhat  by  the 
background  in  which  the  target  appears.  This  Interpretation  is  favored 
because  it  emphasizes  the  larger  role  of  target  type,  than  background, 
in  determining  acquisition  performance. 

The  following  sections  will  consider  in  more  detail  the  nature  and 
interpretation  of  the  main  effects  and  the  locus  of  the  interaction. 


TARGET  EFFECTS 

Given  the  importance  of  target  type,  it  is  useful  to  consider  the 
relative  detectability  of  the  five  targets  used  in  this  study.  Figure  2 
plots  the  mean  detection  times  for  the  various  targets  as  well  as  the 
mean  probability  of  error  which  is  given  by  one  minus  the  probability 
of  correct  detection. * Clearly  the  functions  are  very  much  alike  and 
both  show  wide  differences  in  the  relative  detectabilities  of  the  targets. 


BRIDGE  BRIDGE 
W/AAA  ALONE 


POL  3-TANKS 
SITE 


FIGURE  2.  Mean  Detection  Time  and  Probability 
of  Error  for  Different  Target  Types. 

The  author  recognizes  that  properly  these  data  should  be  plotted  as  , 
histogram,  since  the  abscissa  plots  a non- continuous  variable.  The 
data  were  plotted  as  shown  for  clarity. 
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In  fact  detection  time  and  probability  of  correct  detection  are  atatiati- 
cally  correlated:  the  Pearaon  product -moment  correlation  between 
these  variables  is  -0.  967.  As  a result  of  this  strong  correlation,  this 
and  subsequent  discussions  will  focus  on  the  detection  time  variable, 
assuming  that  the  same  conclusions  apply  to  the  probability  of  detection. 

These  data  were  t-sted  by  a series  of  post  hoc  comparisons 
designed  to  determine  where  significant  differences  in  mean  detecta- 
bility occurred.  Those  differences  were  tested  by  a series  of  pairwise 
comparisons  using  the  Newman- Keula  critical  range  method.  The 
results  of  this  test  were  identical  for  both  dependent  variables.  The 
three  tank  group,  while  appearing  to  be  the  most  difficult  to  detect, 
based  on  Figure  2,  was  not  significantly  different  from  the  POL  site. 
The  POL  site  was  more  difficult  (p  < 0.  01)  than  the  SAM  site,  and  the 
three  tank  group  was  also  more  difficult  (p  < 0.01)  than  the  SAM  site. 
The  bridge  alone  did  not  significantly  differ  from  the  SAM  site,  but 
the  POL  and  the  three  tank  group  were  reliably  more  difficult  than  the 
bridge  alone  (p  <•  0.01  in  both  cases).  The  bridge  with  the  adjacent 
AAA  site  did  not  differ  from  the  bridge  alone  or  SAM  but  was  reliably 
different  from  the  POL  site  and  three  tank  group  (p  s 0.0 1 in  both 
cases). 

In  summary,  the  comparisons  show  that  these  five  targets  fall  into 
two  re  1 in  Id  v different  groups  in  terms  of  mean  detectability.  The  POL 
site  and  three  tank  group  are  not  significantly  different  from  each 
other  and  are  the  most  difficult  targets;  the  bridges  are  about  equal  to 
the  SAM  site  and  are  reliably  easier  to  detect  than  the  tanks  or  POL 
s ite. 

It  would,  ol  course,  be  appropriate  at  this  point  to  provide  a theory 
to  account  for  the  differences  in  the  relative  detectabilities  of  these 
targets.  However,  as  that  is  a goal  of  Phase  II  of  this  effort  it  is 
premature  to  provide  such  a theoretical  framework.  One  way  to  pro- 
ceed is  to  first  consider  the  experimental  situation  and  the  behavior  of 
the  subjects.  In  this  situation  the  subject  is  searching  for  a prebriefed 
target  that  has  been  seen  previously  in  the  training  trials.  It  is  not 
unreasonable  to  conclude  that  from  such  information  the  observer  has 
certain  expectations  as  to  where  a target  is  likely  to  be  located,  much 
as  a pilot  would  have  expectations  about  where  his  prebriefed  targets 
may  be.  Such  expectations  surely  modify  the  observer's  search 
behavior,  by  guiding  the  search  behavior  fo  terrain  areas  that  may 
logically  contain  the  target.  Further,  it  is  likely  that  targets  which 
are  highly  constrained  to  logically  appear  in  particular  scene  loci  will 
be  detected  more  rapidly,  if  only  because  fewer  potential  loci  must  be 
examined.  Conversely,  targets  which,  by  their  nature,  maybe  located 
in  a wide  range  of  scene  locations  will  require  more  search  time. 
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In  the  context  of  this  study,  the  above  scheme  provides  a way  of 
thinking  about  the  relative  detectabilities  of  the  targets.  Bridges  are 
very  constrained  in  their  potential  locations,  always  appearing  on  or 
near  roads  or  rivers  and  consequently  an  observer's  search  strategy 
rejects  inspection  of  areas  without  such  features,  lessening  time  to 
detection.  SAM  sites,  which  did  not  differ  statistically  in  relative 
detectability  from  the  two  bridge  targets,  are  also  constrained,  appear- 
ing chiefly  in  open,  relatively  flat  terrain  areas.  Such  areas  are  few 
in  the  presented  scenes  and  are  easy  to  locate  quickly.  The  POL  site 
and  three  tank  group,  however,  logically  may  occur  nearly  anywhere 
in  a scene  and  thus  require  more  time  to  locate. 

This  admittedly  speculative  scheme  is  proffered  only  as  a way  of 
explaining,  post  hoc,  the  relative  target  detectability  data  discussed 
above.  It  appears  to  provide  a framework  within  which  both  target  type 
and  background  characteristics  may  be  quantitatively  formulated. 

Within  this  framework  it  may  be  possible  to  explain  why  some  targets 
of  equal  visual  subtense  differ  in  detectability  because  of  the  scene 
context  within  which  they  must  occur.  Further,  it  may  provide  an 
account  for  differing  detectability  of  targets  on  different  backgrounds, 
because  they  have  different  potential  areas  within  which  different  target 
types  might  appear,  thus  allowing  an  account  of  target-background 
inte  ractions. 


BACKGROUND  EFFECTS 

Figure  3 shows  a plot  of  mean  detection  time  and  probability  of 
error  as  a function  of  the  six  different  backgrounds.  It  can  be  seen 
that  the  variability  due  to  background  is  less  than  that  due  to  target 
type,  which  is  in  accord  with  the  results  of  the  Eta^  calculations  dis- 
cussed above.  The  statistical  differences  in  these  means  were  also 
tested  by  the  Newman- Keuls  method.  In  contrast  to  the  similar  data 
on  target  type  discussed  above,  the  pattern  of  results  of  this  test  was 
not  in  complete  concordance  for  the  two  dependent  variables;  these  dif- 
ferences, however,  were  minor.  The  time  to  detection  data  showed  no 
significant  differences  among  backgrounds  37,  71,  53,  and  89.  Back- 
ground 37  was  significantly  (p  < 0.0  5)  different  from  background  6,  but 
backgrounds  89,  53,  and  71  did  not  differ  reliably  from  6.  All  back- 
grounds, save  number  6,  were  reliably  different  from  number  8. 

The  probability  of  detection  data  show  that  the  only  significant  dif- 
ference is  that  background  8 is  significantly  easier  than  backgrounds 
37,  53,  and  71  (p  < 0.05)  in  all  cases.  All  other  comparisons  are  not 
statistically  reliable. 

It  is  impossible  at  this  point  to  provide  a theoretical  account  to 
explain  these  differences  in  detail.  But  a first  step  in  this  direction  is 
provided  by  an  informal  classification  of  the  scenes  into  desert 
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FIGURE  3.  Mean  Detection  Time  and  Probability 
of  Error  for  Different  Backgrounds. 

(no's,  6 and  81),  desert/mountain  (8  and  71)  and  rural  (37  and  53) 
terrain  types.  Very  generally,  target  detection  is  best  in  desert  and 
desei  t/mountain  terrain,  and  more  difficult  in  rural  terrain.  However, 
these  conclusions  are  tentative  and  based  only  on  informal,  global 
terrain-type  classification.  A more  comprehensive  account  will  be  given  in 
Phase  II  of  this  contractual  effort. 


INTERACTION 

Figure  4 ihows  plots  of  mean  detection  time  vs.  target  type  with 
background  as  a parameter;  these  data  are  plotted  on  two  axes  for 
clarity.  The  presence  of  a significant  interaction  is  shown  by  the  fact 
that  the  ordering  of  the  relative  detectabilities  of  the  targets,  as  shown 
in  Figure  2,  is  not  obtained  for  all  backgrounds.  Generally,  this  order- 
ing is  preserved  for  backgrounds  8,  53,  and  89  (i.e.,  Figure  4b)  but 
not  tor  6,  37,  and  71  as  in  Figure  4a.  That  detectability  of  the  target 
types  varies  as  a function  of  background  is,  of  course,  simply  another 
way  of  saying  an  interaction  is  present. 
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The  appropriate  analysis  to  conduct  to  isolate  the  locus  of  the 
interaction  effect  is  to  test  for  the  significance  of  simple  main  effects 
o;  one  factor  at  the  various  levels  of  the  other  factor;  this  test  essen- 
tially assesses  the  effect  of  variation  in  one  factor  at  a time.  In  terms 
of  this  experiment,  this  test  determines  if  there  are  significant  differ- 
ences between  backgrounds  at  various  levels  of  the  target  type  factor. 
The  results  of  this  are  presented  in  Table  3.  A separate  error  term 
is  shown  for  each  factor  because  the  appropriate  error  term  is  a 
" within-groups"  mean  square  based  on  scores  of  the  groups  of  subjects 
(i.e.,  the  target-type  scores,  since  the  subjects  factor  is  nested  within 
target  types)  at  each  level  of  background.  The  results  show  a signifi- 
cant variation  due  to  the  background  for  all  target  types  except  the 
SAM  site.  Thus  the  detectability  of  the  SAM  site  is  the  same  over  all 
the  backgrounds  employed  in  this  study,  while  there  are  significant 
differences  in  performance  for  the  other  target  types  as  a function  of 
background.  The  last  column  of  the  table  shows  a calculation  of  an 
Eta^  measure  for  this  comparison.  In  this  case  Eta2  was  computed  by 


TABLE  3.  Analysis  of  Simple  Main  Effect  of  Backgrounds  at 
Different  Target  Types. 


Source 

df 

SS 

MS 

F-Ratio 

Probability 

Level 

Eta2 

BKC  at  Bridge 
w/ AAA 

' * ' ' " 

5 

639. 82 

127,96 

7.25 

P < 0. 001 

0.  1171 

Error  Term 

20 

352.  81 

17.64 

BKG  at  Bridge 

5 

1087. 35 

217.47 

3.05 

P < 0.05 

0. 1990 

Error  Term 

20 

1427.  37 

71. 36 

BKG  at  SAM 

5 

369. 37 

73.87 

0.  939 

NS 

0.0675 

Error  Term 

20 

1 573. 07 

78.  65 

BKG  at  POL 

5 

2598, 86 

519. 77 

8.  51 

P < 0.001 

0. 4756 

Error  Term 

20 

1221. 53 

61.08 

BKG  at  3 -Tank 

5 

754. 37 

150.87 

3.39 

P < 0.025 

0. 1380 

Error  Term 

20 

889. 97 

44.  50 
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dividing  the  sum-of- squares  tor  each  term  by  the  total  sum-of-squares 
tor  the  test  as  a whole.  The  results  of  this  calculation  show  that  most 
of  the  variance,  almost  48c,la,  is  accounted  for  by  the  variation  in  per- 
formance over  background  for  the  POL  target.  This  result  appears 
reasonable  in  light  of  the  variation  in  mean  performance  for  the  POL 
site,  from  a low  of  4.  14  sec  to  detection  for  background  6,  a desert 
scene,  to  a maximum  of  30  sec  for  background  53,  rural  terrain. 

The  complementary  analysis,  the  simple  main  effect  of  target  type 
at  different  background  levels  is  clarified  by  re-plotting  the  data  of 
Figure  4 as  detection  time  against  background  with  target  as  the  param- 
eter; these  plots  are  shown  in  Figure  5,  plotted  on  two  axes  for  clarity. 
Again  the  fact  that  these  plots  do  not  show  the  same  ordering  of  target 
detectability  as  a function  of  background  as  in  Figure  3 simply  demon- 
strates the  presence  of  the  interaction.  The  results  of  the  test  for  the 
simple  main  effect  of  target  at  the  different  backgrounds  is  shown  in 
Table  4.  These  data  show  significant  differences  in  time  to  detection 
for  the  targets  in  all  backgrounds  except  No.  8,  a desert/ mountain 
scene.  The  Eta1-  calculation,  formed  in  the  same  way  as  discussed 
above,  show  that  most  of  the  variance  in  this  test  is  due  to  background 
No.  53,  a rural  scene. 

These  data,  taken  together,  show  that  target  detection  is  influenced 
strongly  both  by  the  target  type  being  searched  for  as  well  as  the  back- 
ground. Adequate  description  of  target  and  background  effects  would 
permit  adequate  predictions  of  such  interactions. 


INFLUENCE  OF  GENERIC  BACKGROUND  TYPES 

Another  way  to  clarify  the  interaction  effects  is  to  collapse  back- 
grounds into  three  'generic'  types;  that  is  scenes  6 and  89  represent 
desert  terrain,  scenes  8 and  71  represent  desert/ mountain  terrain, 
and  rural  terrain  is  represented  by  scenes  37  and  53.  It  must  be 
remembered  that  these  are  informal  classifications  only  for  the  pur- 
pose of  clarifying  the  data,  but  examination  of  the  actual  scenes  in 
Appendix  A shows  that  these  classifications  are  reasonable. 

Figure  6 shows  detection  performance  for  the  five  target  types  as  a 
function  of  the  generic  scene  classification.  Tests  of  the  simple  main 
effect  of  target  with  backgrounds  collapsed  into  these  generic  types 
were  significant  beyond  the  P < 0.01  level  at  all  three  generic  types, 
which  again  shows  the  importance  of  target  type.  However,  Figure  6 
shows  the  large  influence  of  the  rural  scene  on  detectability  of  the 
POL  site.  Since  in  these  scenes  the  POL  site  was  located  along  or 
near  tree  lines,  it  is  not  surprising  this  target  in  these  backgrounds 
was  more  difficult. 
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TABLE  4.  Analysis  of  Simple  Main  Effect  of  Target  Type 
Different  Backgrounds. 


Source 


TGT  at  No.  3 
Error  Term 
TGT  at  No.  6 
Error  Term 
TGT  at  No.  89 
Error  Term 
TGT  at  No.  53 

Error  Term  2 
TGT  at  No.  71 

Error  Term  2 
TGT  at  No.  37 


The  effect  of  background  on  target  type  Is  shown  in  Figure  7.  Here 
simple  tests  of  the  main  effect  of  background  at  various  levels  of  tar- 
get type  revealed  significant  differences  only  for  the  bridge  with  AAA 
site  and  POL  target  (p  < 0.  03,  p < 0.  01,  respectively).  Thus,  the 
other  targets  were  not  significantly  more  or  less  detectable  in  any 
generic  background  type. 


ss 

MS 

F- Ratio 

Probability 

Level 

Eta2 

549.08 

137. 27 

2.07 

N.  S. 

0.0689 

1327.83 

66.  39 

1868.20 

467.05 

8.99 

P < 0.001 

0. 2346 

1039.63 

51. 98 

2909.92 

727.48 

13.62 

P < 0.001 

0. 3650 

1068. 15 

53.41 

4192.88 

1048. 22 

37.33 

P < 0.001 

0,  5265 

561.62 

28.08 

1786.62 

446.62 

4.26 

P < 0.025 

0.  2243 

2098.82 

104.94 

1129.18 

282.29 

3.02 

■ 

0.  1413 

1867.45 

93.37 

1 

GENERALITY  OF  RESULTS 

The  results  presented  to  this  point  have  treated  both  backgrounds 
and  target  types  as  fixed  effects;  that  is,  the  analysis  has  assumed 
that  the  levels  of  these  two  factors  used  in  the  experiments  are  the 
only  ones  of  interest.  Thus,  the  inferences  drawn  from  data  analyzed 
in  this  way  are  confined  only  to  the  levels  of  the  factors  actually 
examined,  obviously  limiting  the  generality  of  the  results.  Testing 
targets  or  backgrounds  on  random  effects,  hov/ever,  is  based  on  the 
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a* sumption  that  the  levels  of  these  factors  are  simply  representative 
of  a larger  population  of  possible  factor  levels.  Analysing  the  data  In 
this  way  permits  answers  to  questions  such  as  whether  there  are  sig- 
nificant differences  In  target  detection  over  all  backgrounds  or  U back- 
ground Influences  target  detection  for  all  types  of  targets  and  not  just 
those  represented  in  this  study. 

The  results  of  this  analysis  are  straightforward.  When  the  target 
factor  Is  analyzed  as  a random  effect,  which  Is  equivalent  to  asking  the 
question  whether  there  is  a significant  background  effect  assuming  these 
targets  are  sampled  from  a larger  population,  a non- significant  F-test 
is  obtained  (F5  20  s 1.8b,  p = 0.  15).  This  result  Implies  that  the  slg  - 
nlflcant  background  effect  obtained  when  targets  a re  a fixed  effect  does 
not  generalize  to  all  possible  targets.  Thus,  the  significant  effect  due 
to  background  in  this  experiment  Is  specific  to  the  targets  used  here. 

The  results  of  the  analysis  with  background  as  a random  effect  were 
different.  A significant  target  effect  (F<tf20  = 11.72,  p<  0,001)  (n 
this  case  implies  that  significant  differences  in  target  detectability 
will  obtain  for  any  background  type  that  is  examined.  This  conclusion 
supports  the  idea  presented  above  that  target  effects  account  for  more 
variance  and,  therefore,  are  relatively  more  important  than  back- 
ground type  in  determining  target  acquisition  performance. 


CUMULATIVE  PROBABILITIES  OF  DETECTION 

A means  for  ovamining  the  joint  results  of  time  to  detection  and. 
probability  of  detection  Is  to  plot  the  cumulative  probability  of  detection 
as  a function  of  time.  Figure  8 shows  cumulative  probabilities  for  the 
various  target  types.  The  terminal  probabilities  show  that  the  bridge 
with  the  AAA  site  is  the  most  likely  to  bo  detected  followed  by  the  bridge 
alone,  SAM  site,  POL  and  finally  the  throe  tank  group.  It  can  be  seen 
that  the  initial  slopes  of  the  two  bridge  targets  and  tho  SAM  site  are 
very  similar,  with  about  60%  or  more  of  correct  detections  occurring 
within  the  Initial  five  seconds.  Interestingly,  the  SAM  site  data  appears 
to  be  shifted  to  the  right  by  about  two  seconds  in  comparison  to  tho 
bridge  targets;  no  SAM  detections  occur  within  the  first  two  seconds, 
while  there  is  a 0,  34  probability  of  detecting  a bridge  and  a 0.  26  pro- 
bability of  detecting  the  bridge  with  the  AAA  site  in  tho  first  two  seconds 
of  search.  This  more  rapid  detection  probably  reflects  the  advantage 
provided  by  locating  3 target  on  or  near  a road,  as  clearly  demonstrated 
in  tho  study  by  Olzak  , Roads  seem  to  be  powerful  cues  to  the  search 
process  and  are  most  likely  the  first  terrain  features  to  be  searched. 

Detection  of  the  POL  and  three  tank  target  is  much  slower  and  less 
likely  to  bo  successful.  Tho  terminal  probability  of  detection  for  tho 
three  tank  group  is  only  3U%,  which  is  much  lower  than  any  other  tar- 
get. Initially,  it  was  thought  that  the  subjects  in  this  group  may  have 
performed  poorly  due  simply  to  random  sampling,  Another  group  of 
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FIGURE  8.  Cumulative  Probability  of  Correct  Detection 
as  a Function  of  Time  for  Different  Target  Types. 


five  subjects  was  run  to  determine  if  this  was  the  case,  but  their  data 
were  virtually  identical  to  the  original  group  and  therefore  it  was  con- 
cluded that  the  original  three  tank  group  data  were  accurate  and  the  new 
data  were  not  used  for  further  analysis. 

The  cumulative  probabilities  of  detection  for  the  three  tank  group 
and  POL  site  seem  to  show  that  for  these  difficult  targets  the  full  30 
seconds  search  time  was  not  utilized.  The  POL  and  tank  group  reach 
asymptotic  lovels  of  detection  probability  after  about  10  and  15  seconds 
respectively:  the  remainder  of  the  search  time  does  not  seem  to  im- 
prove performance.  This  suggests  that  the  initial  few  seconds  of  search 
are  the  most  critical,  and  if  the  observer  has  not  detected  the  target 
within  this  time  there  is  little  likelihood  of  doing  so  given  more  time, 
This  result  has  implications  for  modeling  target  acquisition  perfor- 
mance in  that  the  performance  during  the  first  few  seconds  of  search 
ought  to  bo  a good  predictor  of  terminal  performance  and  that  modeling 
ought  to  concentrate  on  this  time  period. 
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FIGURE  9.  Cumulative  Probability  of  Correct 
Detection  as  a Function  of  Time  for  Generic 
Background  Types. 


The  cumulative  probabilities  of  detection  for  the  generic  backgrounds, 
rural,  desert,  and  desert/mountain,  are  presented  in  Figure  9*  Here  it 
can  be  seen  that  the  plots  are  roughly  similar  for  the  desert  and  desert/ 
mountain  types,  but  the  rural  scenes  have  lower  initial  slopes  and 
lower  final  probability  values.  It  is  interesting  to  note  that  in  the  plot 
for  the  rural  scenes  the  contributions  of  the  two  scenes  (37  and  53)  is 
somewhat  different  in  that  the  maximum  detection  time  for  any  target 
in  scene  53  was  about  5 seconds.  All  data  in  the  composite  plot  beyond 
5 seconds  was  contributed  only  by  scene  37.  It  is  not  clear  at  this 
point  why  one  rural  scene  permitted  much  more  rapid  detection;  this 
will  be  investigated  more  fully  in  the  modeling  phase. 

''The  data  points  in  this  figure  were  omitted  for  clarity.  The  curves 
are  tracings  of  the  data  points. 
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COMPARISON  TO  OLZAK'S  STUDY 

The  design  and  procedures  of  the  Qlzak^  study  were  essentially  the 
same  as  those  of  the  present  study.  This  permits  an  assessment  of 
the  relative  strength  of  target  and  background  effects  when  a single 
target  type  is  presented  in  different  numbers  and  configurations,  i.e.  , 
Olzak's  study,  in  comparison  to  the  present  study,  which  used  funda- 
mentally different  target  types.  In  the  Olzak  study  there  were  three 
within  group  variables:  scene  complexity,  complexity  replications, 
and  local  context.  For  the  purposes  of  comparison  to  the  present 
study,  these  variables  wore  collapsed  to  form  a single  within  group 
variable  of  background  type.  An  ANOVA  was  performed  on  Olzak's 
data,  and  the  results  are  shown  in  Tables  5 and  b for  time  and  prob- 
ability of  detection  data,  respectively. 

The  re-analysis  of  Olzak's  data  shows  that  both  main  effects  and  the 
interaction  are  highly  significant  for  both  dependent  measures.  This, 
of  course,  is  exactly  the  same  overall  result  as  in  the  present  expuri- 
munt.  A difference  occurs  when  the  Eta’  calculations  arc  examined. 

The  present  study  showed  that  the  target  effect  accounts  f*>r  up  to  five 
times  as  much  variance  as  the  background  effect.  Olzak's  data,  on  the 
other  hand,  show  that  background  effect  contributes  slightly  more  (for 
the  time  data)  or  just  slightly  less  (for  probability  data)  variance  than 
the  target  effect.  This  difference  is  not  attributable  to  the  overall 
variance  accounted  for  in  the  experiment,  which  is  comparable  for 
both  dependont  variables  between  the  two  studies.  That  is,  • time  and 
probability  accounted  for  64%  and  52%,  respectively,  in  this  experi- 
ment versus  60%  and  49%  in  Olzak's  study.  In  any  case,  the  difference 
lies  in  the  apportionment  between  the  main  effects  and  interaction  of 
the  total  variance  accounted  for  in  the  experiment. 

This  difference  reflects  the  contrasting  emphasis  between  the  two 
studies.  It  seems  reasonable  to  conclude  that  a single  target  type,  in 
this  case  a tank,  differing  only  in  number  and  configuration,  would 
account  for  less  overall  experimental  variance  than  target  types  differ- 
ing fundamentally  in  kind,  as  well  as  number  and  configuration.  This 
is  precisely  the  result  shown  by  comparisons  of  the  Eta"  values  between 
both  experiments. 

That  background  accounted  for  more  variance  in  OUak's  study  may 
also  be  due  to  the  fact  that  12  backgrounds  were  used  in  that  study  as 
opposed  to  only  six  in  this  one.  But  the  fact  that  background  accounts 
for  just  about  twice  as  much  variance  in  Olzak*  s study  as  the  present 
one  is  probably  coincidental. 

A final  reason  for  the  smaller  contribution  of  background  to  the 
overall  experimental  variance  in  the  present  study  lies  in  selection  of 
backgrounds.  Five  of  the  six  backgrounds  in  this  study  were  used  in 
Olzak' s,  and  these  five  were  contrasted  to  the  other  seven  in  that 
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TABLE  6.  Analysis  of  Variance  Summary:  Probability  of  Detection 
Reanalysis  of  Data  from  Glzak,  1978. 
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studv.  The  mean  time  to  detection,  collapsed  over  all  target 
i onfiguratiims,  tor  the  five  used  in  this  study  wara  13.7  sec  versus 
10.74  set.-  for  the  other  seven.  This  difference  was  tested  as  a con- 
trast on  marginal  means  by  an  F-test  and  the  obtained  ratio  was 
Fj,  20  - 4.  74  (P  < 0. 05).  Therefore,  the  backgrounds  used  in  this 
experiment  were  significantly  more  difficult  than  the  othe  r backgrounds 
from  Olzak' s study.  It  is  likely  that  these  five  are  more  homogenous 
in  terms  of  the  variance  they  contribute,  and  therefore  it  is  not 
unusual  that  the  overall  variance  contributed  by  background  is  less  in 
the  present  study. 

It  seems,  however,  that  the  difference  in  the  proportion  of  variance 
accounted  for  by  target  and  background  between  the  two  studies  does 
reflect  an  important  consideration  for  target  acquisition  modeling, 
namely  that  background  type  is  more  important  in  determining  perfor- 
mance when  a single  target  type  in  under  consideration,  than  when 
multiple  target  types  are  being  considered.  The  relative  effects  of 
background  must  be  weighted  to  a greater  or  lesser  degree,  depending 
on  what  purposes  a model  is  being  exercised. 

An  operational  situation  of  interest  illustrating  this  point  might  be 
one  in  which  a target  acquisition  model  is  being  used  to  compute  the 
weapon  resources  to  be  used  against  tactical  targets.  If,  based  on 
other  intelligence,  the  mission  planner  knows  that  the  strike  is  to  be 
only  against  tank  formations,  then  the  background  component  of  the 
model  ought  to  be  given  greater  weighting  than  if  a mission  is  being 
planned  against  a variety  of  tactical  targets.  It  remains  an  empirical 
question,  however,  to  determine  if  target  acquisition  modeling  is 
significantly  improved  by  a differential  weighting  of  background  effects, 
and  if  so,  whether  the  improvement  is  justified  by  the  increased  cost 
and  additional  complexity  of  such  a weighting  scheme. 

A final  point  of  comparison  between  the  two  studies  regards  the 
relative  performance  of  the  three  tank  groups  in  the  two  studies.  The 
cumulative  probabilities  for  the  three  tank  group  in  this  study  are  very 
much  different  than  those  of  the  Olzak  study,  with  the  final  probability 
of  detection  levels  being  30%  in  this  study  as  opposed  to  about  70%  in 
the  previous  study.  This  difference  probably  reflects  differences  in 
the  target  placement  within  the  scenes,  as  a conscious  effort  was  made 
in  the  present  study  to  place  the  three  tank  group  in  different  locations 
within  a scene  than  in  the  Olzak  study.  However,  until  better  metrics 
are  developed  relating  the  way  targets  interact  with  backgrounds, 
further  speculations  on  this  difference  is  best  suspended. 
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CONCLUSIONS 


The  data  obtained  in  this  experiment  serve  to  reinforce  the  notion  that 
target  and  scene  characteristics  and  their  interaction  are  critical  in 
determining  target  acquisition  performance.  As  such,  these  character- 
istics require  a more  quantitative  formulation  which  expresses  the  im- 
portance of  the  interactive  effects  of  target  type  and  background  char- 
acteristics. However,  as  these  data  are  intended  to  be  incorporated  into 
a data  base  which  will  permit  the  later  development  of  such  a quantita- 
tive formulation,  it  iu  not  the  purpose  of  this  interim  report  to  provide 
a theory  to  account  for  the  observed  effects.  The  following  paragraphs 
will  briefly  summarize  the  qualitative  aspects  and  implications  of  the 
data  obtained  in  this  summary. 


TARGET  EFFECTS 

By  far  the  moat  important  factor  in  determining  the  level  of  target 
acquisition  performance  is  target  type.  Statistical  analyses  showed 
that  the  five  target  types  used  in  this  study  fell  into  two  reliably  differ- 
ent groups  in  terms  of  relative  detectability.  The  bridge  target,  the 
bridge  with  an  adjacent  AAA  site,  and  the  SAM  site  were  reliably  easier 
to  locate  than  the  POL  supply  dump  or  three  tank  group.  The  mean 
time  to  detection  and  probability  of  detection  for  these  groups  were, 
respectively,  7.  74  seconds  and  0.88  probability  versus  IZ.  58  seconds 
and  0,  39  probability.  One  way  to  account  for  these  results  is  to  con- 
sider that  the  detection  time  for  a specific  target  is  inversely  related 
to  the  degree  of  constraints  on  potential  location  of  the  target.  That  is. 
targets  which  are  highly  constrained  to  appear  in  certain  scene  con- 
texts will  have  shorter  detection  times.  This  effect  is  perhaps  due  to 
the  observer's  expectation  of  potential  target  locations  modifying 
search  behavior.  It  would  be  highly  desirable  to  test  this  notion  directly 
by  monitoring  eye  movements  during  search  with  these  or  similar  tar- 
get types. 


BACKGROUND  EFFECTS 

Background  tvpe  unexpectedly  accounted  for  a small  proportion  of 
the  variance  in  this  experiment  in  comparison  to  another  related  study^. 
However,  this  result  is  obtained  because  widely  different  target  types 
were  being  compared  here  as  opposed  to  a single  target  type  differing 
in  number  and  configuration.  This  result  also  implies  that  prior 
knowledge  of  target  type  subtly  modifies  the  influence  of  scene  charac- 
teristics on  search  behavior,  but  the  nature  of  this  modification  is 
unclear  and  cannot  directly  be  addressed  in  experiments  of  this  kind. 
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IMPLICATIONS  FOR  MODELING 

Those  results  have  some  important  implications  for  the  formulation 
of  predictive  models  of  target  acquisition.  First,  target  type  will  have 
to  bo  quantified  in  some  way  other  than  a simple  size  or  number 
specification.  For  example,  if  the  number  of  elements  comprising 
the  target  was  the  only  quantification  of  target  type,  the  SAM  site, 
which  contained  8-10  elements,  should  have  been  the  easiest  target 
to  find,  while  the  bridge  alone  should  have  been  most  difficult,  results 
which,  however,  did  not  obtain.  A quantification  of  target  type  ought 
to  include  the  notion  that  different  targets  differ  fundamentally  in  the 
scene  contexts  within  which  they  might  be  found. 

Background  effects,  while  Important,  seem  to  be  outweighed  by 
target  effects  when  widely  differing  targets  are  being  compared.  This 
implies  that  the  weighting  given  to  background  effects  in  a model 
cannot  be  constant  but  must  vary  according  to  what  comparisons  are 
desired. 

Further,  since  background  and  target  type  interact  strongly,  a use- 
ful quantification  of  terrain  might  be  one  which  considers  the  number 
of  potential  locations  for  a particular  target,  rather  than  a global 
measure  of  the  overall  characteristics  of  a scene. 

Target  acquisition  is  a very  complex  behavior,  and  any  single 
experiment  cannot  fully  account  for  all  the  variations  in  performance. 
The  present  results  do  suggest  some  intriguing  effects  and  some 
potentially  useful  ways  to  think  about  this  complex  behavior. 


IMPLICATIONS  FOR  OPERATIONS 

An  important  use  of  target  acquisition  models  is  determining  the 
allocation  of  minimum  weapon  resources  to  be  used  in  a tactical  strike 
to  achieve  a given  probability  of  success.  The  present  results,  at  this 
stage  of  development,  do  not  permit  quantitative  answers  to  such 
questions.  However,  in  a qualitative  way  these  data  show  that  because 
the  probability  of  error  and  the  time  to  detection  are  less  for  the 
bridge  targets  and  the  SAM  site,  in  comparison  to  the  POL  and  three- 
tank  target,  fewer  resources  might  be  allocated  for  a stike  against 
the  former  targets.  The  results  alno  show  that  targets  located  in 
rural  terrain  would  require  more  resources  than  those  located  in  desert 
or  desert /mountain  terrain.  These  conclusions  assume  that  resource 
allocation  is  determined  by  the  time  or  probability  of  detection.  This, 
of  course,  is  an  ove rsimplication.  Further  speculations  on  how  these 
data  may  ultimately  be  used,  however,  are  well  beyond  the  scope  of  this 
report. 
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